Apparatus and method for generating panoramic sound

ABSTRACT

An apparatus and method for generating a panoramic sound are provided. The panoramic sound generation apparatus may include a panning coefficient calculation unit to calculate a panning coefficient that represents directivity of a sound source using an input signal, a masker determination unit to determine a direction masker that extracts a sound source of a desired direction based on the panning coefficient, and a channel separation unit to separate to be used as an output signal, output to a sound output device, using the direction masker.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority benefit of Korean PatentApplication No. 10-2012-0000070, filed on Jan. 2, 2012, in the KoreanIntellectual Property Office, the disclosure of which is incorporatedherein by reference.

BACKGROUND

1. Field

Embodiments of the following description relate to an apparatus andmethod for generating a panoramic sound and, more particularly, to anapparatus and method for generating a more realistic panoramic sound byapplying signal processing to a 2-channel stereo input so that a soundsurrounds a listener.

2. Description of the Related Art

Conventionally, audio contents are provided to a listener in a 2-channelstereo format. As a multichannel system is recently becoming widespread,a demand from users for multichannel signal reproduction or amultichannel signal effect is increasing rapidly.

Reproduction of a multichannel signal may be performed by converting anoriginal 2-channel signal to a multichannel signal and reproducing themultichannel signal via a multichannel speaker. Here, a technology forexpanding an input 2-channel to a multichannel is called up-mixing.However, when a real speaker is adapted for only a 2-channel signal,post-processing of an output signal is necessary to provide a panoramicsound effect to a listener even with the 2-channel speaker.

According to a conventional method for generating a panoramic sound,signal processing is applied to a general 2-channel stereo input signalso that a sound surrounds a listener and a more vibrant sound isprovided to the listener. However, the conventional method has thefollowing problems.

Since the conventional method generates a virtual speaker withoutseparation of a sound source, interference between sound sources hindersgeneration of the virtual speaker.

In addition, a sound may be reproduced in a direction unintended by asound source manufacturer and one sound source may be reproduced throughboth a real speaker and a virtual speaker, thereby causing a ghost soundimage.

Furthermore, in an environment for reproducing an extracted multichannelsignal, a location of a speaker may differ according to users. Thus,since there is a change in a location of the speaker, from an originalintention, a sound image may be lost.

In conclusion, the conventional method of generating a panoramic soundhas the following problems of (i) generation of a ghost sound sourcecaused by overlapping of sound sources, (ii) hindrance in generation ofa virtual speaker due to interference by the overlapping of soundsources, and (iii) damage to a sound image caused when structures aredifferent between a speaker used by a user and a speaker intended by achannel separation technology. Accordingly, there is a need for solvingthe aforementioned limitations.

SUMMARY

According to an aspect of one or more embodiments, there is provided apanoramic sound generation apparatus including a panning coefficientcalculation unit to calculate a panning coefficient that representsdirectivity of a sound source using an input signal, a maskerdetermination unit to determine a direction masker that extracts a soundsource of a desired direction based on the panning coefficient, and achannel separation unit to separate the input signal to be used as anoutput signal, and to output the output signal to a sound output device,using the direction masker.

The panoramic sound generation apparatus may further include a virtuallocalization unit to generate a virtual signal from the output signalaccording to a type of the sound output device.

According to an aspect of one or more embodiments, there is provided apanoramic sound generation method including calculating a panningcoefficient that represents directivity of a sound source using an inputsignal; determining a direction masker that extracts a sound source of adesired direction based on the panning coefficient; and separating theinput signal to be used as an output signal, and outputting the outputsignal to a sound output device, using the direction masker.

The panoramic sound generation method may further include generating avirtual signal from the output signal according to a type of the soundoutput device.

According to an aspect of one or more embodiments, there is provided apanoramic sound generation apparatus including a masker determiner todetermine a direction masker that extracts a sound source of a desireddirection based on a panning coefficient which represents directivity ofa sound source based on an input signal; and a channel separator toseparate the input signal to be used as an output signal, and to outputthe output signal to a sound output device, using the direction masker.

According to an aspect of one or more embodiments, there is provided apanoramic sound generation method including determining a directionmasker that extracts a sound source of a desired direction based on thepanning coefficient which represents directivity of a sound source basedon an input signal; and separating the input signal to be used as anoutput signal using at least one processor, and outputting the outputsignal to a sound output device, using the direction masker.

According to another aspect of one or more embodiments, there isprovided at least one non-transitory computer readable medium storingcomputer readable instructions to implement methods of one or moreembodiments.

A panoramic sound generation apparatus according to embodiments mayprovide a realistic sound effect to a user.

A panoramic sound generation apparatus may extract panning informationof a sound source based on a 2-channel input signal, and may separatelysupply the sound source according to a location of a speaker being usedby the user based on the panning information. Therefore, a ghost soundsource caused by overlapping of sound sources may be removed.

Since panoramic sound generation apparatus may apply a sound sourceseparation technology, overlapping of sound sources at one speaker isprevented. Accordingly, interference by overlapping of sound sourcesbetween speakers may be prevented.

A sound output device receives configuration information of a speakerfrom a user and extracts a sound source corresponding to a direction ofa speaker currently being used. Therefore, damage of a stereo imagecaused by inconsistency in directivity between a real speaker and aseparated sound source may be prevented.

BRIEF DESCRIPTION OF THE DRAWINGS

These and/or other aspects will become apparent and more readilyappreciated from the following description of embodiments, taken inconjunction with the accompanying drawings of which:

FIG. 1 illustrates a panoramic sound generation apparatus according toembodiments;

FIG. 2 illustrates a process of deriving an output signal to which apanoramic sound effect is applied according to embodiments;

FIG. 3 illustrates distribution of a panning coefficient in atime-frequency (T-F) grid according to embodiments;

FIG. 4 illustrates a process of calculating a panning coefficientaccording to embodiments;

FIG. 5 illustrates a difference in derived panning coefficientsaccording to a panning coefficient calculating method according toembodiments;

FIG. 6 illustrates a process of determining a direction masker accordingto embodiments;

FIG. 7 illustrates a sound output device, determined by a user,according to embodiments;

FIG. 8 illustrates a process of processing a panning coefficient windowaccording to embodiments;

FIG. 9 illustrates a detailed structure of a channel separation unitaccording to embodiments;

FIG. 10 illustrates a detailed structure of a virtual localization unitwhen a sound output device is a headphone;

FIG. 11 illustrates a detailed structure of a virtual localization unitwhen a sound output device is a stereo speaker; and

FIG. 12 illustrates a panoramic sound generation method according toembodiments.

DETAILED DESCRIPTION

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings, wherein like referencenumerals refer to the like elements throughout. Embodiments aredescribed below to explain the present disclosure by referring to thefigures.

FIG. 1 illustrates a panoramic sound generation apparatus 100 accordingto embodiments.

Referring to FIG. 1, the panoramic sound generation apparatus 100 mayinclude a panning coefficient calculation unit (panning coefficientcalculator) 101, a masker determination unit (masker determiner) 102,and a channel separation unit (channel separator) 103. The panoramicsound generation apparatus 100 may further include a virtuallocalization unit (virtual localizer) 104.

The panning coefficient calculation unit 101 may calculate a panningcoefficient representing directivity of a sound source using an inputsignal. For example, the panning coefficient calculation unit 101 maycalculate the panning coefficient using a frequency component based onthe input signal.

The masker determination unit 102 may determine a direction masker forextracting a sound source of a desired direction based on the panningcoefficient. Here, the masker determination unit 102 may determine thedirection masker for masking a sound source having a panning coefficientcorresponding to a direction of a sound output device that correspondsto a target location.

For example, the masker determination unit 102 may include a controlcoefficient determiner and a window processor.

The control coefficient determiner 102 may determine a controlcoefficient for windowing a panning coefficient using configurationinformation of the sound output device. For example, the controlcoefficient determiner may determine the control coefficient using afirst angle based on a first sound output device corresponding to a leftboundary and a second sound output device corresponding to a rightboundary, a second angle based on a third sound output devicecorresponding to a target location, a third angle based on a fourthsound output device adjoining a left side of the target location, and afourth angle based on a fifth sound output device adjoining a right sideof the target location.

The window processor may process a panning coefficient window based onthe sound output device, using the control coefficient. The windowprocessor may process a window of which a value is maximized in adirection corresponding to the sound output device and decreased in adirection toward an adjacent sound output device.

The channel separation unit 103 may separate the input signal to be usedas an output signal, output to a sound output device, using thedirection masker. For example, the channel separation unit 103 may applythe direction masker to the input signal based on an angle of a soundoutput device corresponding to the target location.

The virtual localization unit 104 may generate a virtual signal from theoutput signal according to a type of the sound output device. Forexample, the virtual localization unit 104 may generate the virtualsignal by applying a head related transfer function (HRTF) when thesound output device is a pair of headphones. When the sound outputdevice is a stereo speaker, the virtual localization unit 104 maygenerate the virtual signal by applying the HRTF and a crosstalkcanceller.

The respective elements illustrated in FIG. 1 will be described infurther detail with reference to FIGS. 2 to 9.

FIG. 2 illustrates a process of deriving an output signal to which apanoramic sound effect is applied according to embodiments.

Referring to FIG. 2, an input signal may be transformed from a timedomain signal to a frequency domain signal through a time-to-frequency(T-F) domain transform unit 201. Here, presuming that the input signalincludes a left channel signal S_(L)(t) and a right channel signalS_(R)(t), the left channel signal S_(L)(t) and the right channel signalS_(R)(t) may be transformed to S_(L)(m, k) and S_(R)(m, k),respectively, through the T-F domain transform unit 201. Here, m denotesa frame index and k denotes a frequency index.

According to FIG. 2, a panning coefficient calculation unit 202, amasker determination unit 203, and a channel separation unit 204constitute an acoustic panorama filter.

The transformed left channel signal S_(L)(m, k) and the transformedright channel signal S_(R)(m, k) may be input to the panning coefficientcalculation unit 202. The panning coefficient calculation unit 202 maycalculate a panning coefficient Γ(m,k) using the transformed channelsignals S_(L)(m, k) and S_(R)(m, k). The panning coefficient Γ(m,k) maybe input to the masker determination unit 203. The masker determinationunit 203 may determine a direction masker ψ(m, k) for separating soundsources corresponding to respective sound output devices using thepanning coefficient and configuration information of the sound outputdevice, the configuration information input by a user.

When the sound output device to which a final output signal is to beoutput is not a multichannel speaker or is a headphone, theconfiguration information of the sound output device may includeconfiguration information related to a standard 5.1-channel speaker.

The direction masker ψ(m, k) derived by the masker determination unit203 may be input to the channel separation unit 204. The channelseparation unit 204 may separate, from the transformed input signalsS_(L)(m, k) and S_(R)(m, k), multichannel output signals D₁(m,k), . . ., D_(N)(m,k) to be output from speakers which are the sound outputdevices designated by the user, using the direction masker ψ(m, k).

When the sound reproduction device is the multichannel speaker havingmore than 2 channels, the separated multichannel output signal may betransformed to an output signal of a time domain through afrequency-to-time (F-T) domain transform unit 206. The output signalseparated through the channel separation unit 204 may be output througha corresponding speaker of the multichannel speaker.

When the power reproduction device is a 2-channel speaker or aheadphone, the separated multichannel output signal may be input to thevirtual localization unit 205. The virtual localization unit 205 mayoutput a virtual signal V_(L)(m,k) corresponding to a left channel and avirtual signal V_(R)(m,k) corresponding to a right channel so that asound of the multichannel output signal is audible to the user in avirtual multichannel direction. The virtual signals V_(L)(m,k) andV_(R)(m,k) may be transformed to signals v_(L)(t) and v_(R)(t)corresponding to the time domain through the F-T domain transform unit206, and output through the stereo speaker or a pair of headphones.

The panoramic sound generation apparatus according to embodiments mayprovide a realistic sound effect to a user.

First, the panoramic sound generation apparatus extracts panninginformation of a sound source based on a 2-channel input signal, andseparately supplies the sound source according to a location of aspeaker being used by the user based on the panning information.Therefore, a ghost sound source caused by overlapping of sound sourcesmay be removed.

Second, since the panoramic sound generation apparatus applies a soundsource separation technology, overlapping of sound sources of differentsound sources at one speaker may be prevented. Accordingly, interferencecaused by overlapping of sound sources between speakers may beprevented.

Third, the sound output device receives configuration information of aspeaker from a user and extracts a sound source corresponding to adirection of a speaker being currently used. Therefore, damage of astereo image caused by inconsistency of direction between a real speakerand a separated sound source may be prevented.

FIG. 3 illustrates distribution of a panning coefficient in a T-F gridaccording to embodiments.

The T-F domain transform described with reference to FIG. 2 is performedin units of frames which include predetermined samples. Here, when mdenotes a frame index and k denotes a frequency index, the T-F grid maybe generated as shown in FIG. 3.

It is presumed that, when at least two sound sources are mixed in 2channels, the at least two sound sources are not located on the same T-Fgrid. In this case, the panning coefficient may be calculated withreference to a frequency bin on the T-F grid having the indexes m and k.

In embodiments, it is presumed that the sound source is located at theright channel as the panning coefficient is approximated to 1 and at theleft channel as the panning coefficient is approximated to 0. Inaddition, as the panning coefficient is approximated to 0.5, it ispresumed that the sound source is almost spaced with respect to locatedat both the left channel and the right channel, that is, the soundsource is located in a center with respect to the user.

In FIG. 3, on the T-F grid, the panning coefficient is displayed to bedarker as the panning coefficient is greater and is displayed to belighter as the panning coefficient is smaller. Also, in FIG. 3, as thefrequency index k is greater, the sound source is located at the righton the average and, as the frequency index k is smaller, the soundsource is located at the left on the average.

FIG. 4 illustrates a process of calculating a panning coefficientaccording to embodiments.

The panning coefficient representing the directivity of the sound sourcemay be derived through Equation 1 and Equation 2 introduced below. Thepanning coefficient calculated by Equation 1 may be derived throughconfiguration of a panning coefficient calculation unit shown in FIG. 4.

$\begin{matrix}{{\Gamma \left( {m,k} \right)} = \frac{{S_{R}\left( {m,k} \right)}}{{{S_{L}\left( {m,k} \right)}} + {{S_{R}\left( {m,k} \right)}}}} & \left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack \\{{\Gamma \left( {m,k} \right)} = {\frac{2}{\pi}{\arctan \left( \frac{{S_{R}\left( {m,k} \right)}}{{S_{L}\left( {m,k} \right)}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack\end{matrix}$

Complexity of the panning coefficient is lower when Equation 1 is usedthan when Equation 2 is used. However, the panning coefficientcalculated by Equation 1 has lower accuracy than when the panningcoefficient is calculated by Equation 2 and, therefore, may notcorrectly reflect the direction of the sound source.

FIG. 5 illustrates a difference in derived panning coefficientsaccording to a panning coefficient calculating method according toembodiments.

FIG. 5 shows a difference between the panning coefficient calculated byEquation 1 and the panning coefficient calculated by Equation 2. In agraph shown in FIG. 5, presuming that −90° denotes the left side and+90° denotes the right side, it is understood that the panningcoefficient calculated by Equation 2 reflects the direction of the soundsource more accurately than the panning coefficient calculated byEquation 1. That is, the panning coefficient calculated by Equation 1may cause an error according to the direction of the sound source andtherefore have a lower accuracy than the panning coefficient calculatedby Equation 2.

Accordingly, Equation 1 may be applied to a system requiring quickprocessing while Equation 2 may be applied to a system requiring highquality performance.

FIG. 6 illustrates a process of determining a direction masker accordingto embodiments.

The direction masker is used for extracting a sound source of a desireddirection. For example, to extract a sound source corresponding to thecenter at which the panning coefficient is 0.5, the direction masker ofa T-F grid having a value of 0.5 as the panning coefficient needs to bea value of 1. However, since the sound source is not perfectlyindependent in each T-F grid, the direction masker values in which thepanning coefficient is approximately equal to 0.5 may also be included.

Furthermore, a sound source located between adjacent speakers does notperfectly correspond to a direction of a real speaker. For the soundsource located between the adjacent speakers, the adjacent speakers mayoutput sound simultaneously with a proper sound pressure so that aneffect as if the sound source was reproduced in a location between twoadjacent speakers is achieved.

To achieve the aforementioned effect, a section in which adjacentpanning coefficient windows overlap is necessary. Therefore, thepanoramic sound generation apparatus may be configured so that a sum ofthe two panning coefficient windows becomes 1 in the overlap section.Therefore, loss of any sound source during extraction of the soundsource may be prevented.

FIG. 6 illustrates the process of determining the direction maskersatisfying the foregoing characteristics. When the output signal isoutput to the stereo speaker or the pair of headphones, that is, thesound output device requiring virtual localization, a maskerdetermination unit 601 may determine a control coefficient for windowingthe panning coefficient using configuration information of apredetermined sound output device. The masker determination unit 601 mayalso be referred to as a direction masker calculator. The masterdetermination unit 601 includes a panning coefficient windowingcontroller 602 and panning coefficient windowing 603.

When the output signal is output to a real multichannel speaker, thatis, the sound output device not requiring virtual localization, themasker determination unit 601 may determine a control coefficient forwindowing the panning coefficient by receiving configuration informationof the multichannel speaker which is the sound output device used by theuser. The process of determining the direction masker ψ(m, k) shown inFIG. 6 will be described in detail with reference to FIG. 7.

FIG. 7 illustrates a sound output device, for example a speaker,determined by a user, according to embodiments.

Configuration of the sound output device, input to the direction maskerdetermination unit 601, is shown in FIG. 7. The configurationinformation may be input by the user or may be configuration informationof a predetermined multichannel speaker, for example, a 5.1-channelspeaker.

An angle formed between a speaker 701 corresponding to a left boundarywith respect to the user and a speaker 705 corresponding to a rightboundary is denoted by θ_(max). An angle defined by a speaker 703corresponding to a target location is denoted by θ_(T). An angle definedby an adjacent left speaker 702 with respect to the target location isdenoted by θ_(L). An angle based on an adjacent right speaker 704 withrespect to the target location is denoted by θ_(R).

Control coefficients Γ_(L), Γ_(R), δ_(L), and δ_(R) for windowing thepanning coefficient may be determined by Equation 3 or Equation 4. Here,Equation 3 is used when the panning coefficient is calculated byEquation 1. Equation 4 is used when the panning coefficient iscalculated by Equation 2.

$\begin{matrix}{{{\theta_{T}^{\prime} = {\frac{\pi}{2}\frac{\theta_{T}}{\theta_{\max}}}},{\theta_{L}^{\prime} = {\frac{\pi}{2}\frac{\theta_{L}}{\theta_{\max}}}},{\theta_{R}^{\prime} = {\frac{\pi}{2}\frac{\theta_{R}}{\theta_{\max}}}}}{\rho_{T} = \frac{\sin \; \theta_{T}^{\prime}}{{\cos \; \theta_{T}^{\prime}} + {\sin \; \theta_{T}^{\prime}}}}{\rho_{L} = \frac{\sin \; \theta_{L}}{{\cos \; \theta_{L}^{\prime}} + {\sin \; \theta_{L}^{\prime}}}}{\rho_{R} = \frac{\sin \; \theta_{R}^{\prime}}{{\cos \; \theta_{R}^{\prime}} + {\sin \; \theta_{R}^{\prime}}}}{\delta_{L} = {\rho_{T} - \rho_{L}}}{\delta_{R} = {\rho_{R} - \rho_{T}}}{\Gamma_{L} = {\rho_{T} - \delta_{L}}}{\Gamma_{R} = {\rho_{T} - \delta_{R}}}} & \left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack \\{{{\theta_{T}^{\prime} = {\frac{\pi}{2}\frac{\theta_{T}}{\theta_{\max}}}},{\theta_{L}^{\prime} = {\frac{\pi}{2}\frac{\theta_{L}}{\theta_{\max}}}},{\theta_{R}^{\prime} = {\frac{\pi}{2}\frac{\theta_{R}}{\theta_{\max}}}}}{\rho_{T} = {\frac{2}{\pi}{\arctan \left( \frac{\sin \; \theta_{T}^{\prime}}{\cos \; \theta_{T}^{\prime}} \right)}}}{\rho_{L} = {\frac{2}{\pi}{\arctan \left( \frac{\sin \; \theta_{L}}{\cos \; \theta_{L}^{\prime}} \right)}}}{\rho_{R} = {\frac{2}{\pi}{\arctan \left( \frac{\sin \; \theta_{R}^{\prime}}{\cos \; \theta_{R}^{\prime}} \right)}}}{\delta_{L} = {{\rho_{T} - {\rho_{L}\delta_{R}}} = {{\rho_{R} - {\rho_{T}\Gamma_{L}}} = {{\rho_{T} - {\delta_{L}\Gamma_{R}}} = {\rho_{T} - \delta_{R}}}}}}} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

The direction masker ψ(m, k) for masking a sound source having a panningcoefficient corresponding to a direction of the speaker 703 thatcorresponds to the target location may be determined by Equation 5,using the control coefficients calculated by Equation 3 or Equation 4.

$\begin{matrix}{{\Psi (\Gamma)} = \left\{ \begin{matrix}{0.5\left( {1 - {\cos \left( \frac{\pi \left( {\Gamma - \Gamma_{L}} \right)}{\delta_{L}} \right)}} \right)} & {\Gamma_{L} < \Gamma \leq {\Gamma_{L} + \delta_{L}}} \\{0.5\left( {1 - {\cos \left( \frac{\pi \left( {\Gamma - \Gamma_{R}} \right)}{\delta_{R}} \right)}} \right)} & {{\Gamma_{L} + \delta_{L}} < \Gamma \leq {\Gamma_{R} + {2\delta_{R}}}} \\0 & {otherwise}\end{matrix} \right.} & \left\lbrack {{Equation}\mspace{14mu} 5} \right\rbrack\end{matrix}$

FIG. 8 illustrates a process of processing a panning coefficient windowaccording to embodiments, and shows left boundary speaker 801, adjacentleft speaker 802, target speaker 803, adjacent right speaker 804, andright boundary speaker 805.

When the angle θ_(max) is 180°, the angle θ_(L) is 60°, the angle θ_(T)is 120°, and the angle θ_(R) is 155° in FIG. 7, the panning coefficientwindow may be derived as shown in FIG. 8. Referring to FIG. 8, thepanning coefficient window becomes 1 in a direction corresponding to thespeaker 803 as the sound output device, and is reduced to 0 in adirection toward the adjacent speakers 802 and 804.

The panning coefficient windows may be overlapped between the adjacentspeakers and, as a result, the sum of the panning coefficient windowsbecomes 1, thereby preventing loss of the sound source between adjacentspeakers.

FIG. 9 is a diagram illustrating a detailed structure of a channelseparation unit 901 according to embodiments.

When the angle θ′_(T) of the speaker corresponding to the targetlocation is greater than π/4, the channel separation unit 901 may outputan output signal D(m, k) separated by multiplying a direction masker bya right input signal S_(R)(m, k). Conversely, when the angle θ′_(T) ofthe speaker corresponding to the target location is smaller than π/4,the channel separation unit 901 may output an output signal D(m, k)separated by multiplying a direction masker by a left input signalS_(L)(m, k).

FIG. 10 illustrates a detailed structure of a virtual localization unitwhen a sound output device is a pair of headphones.

When the sound output device is the pair of headphones, the virtuallocalization unit may apply HRTFs 1001 to 1004 in locations T1, T2, . .. etc. of respective predetermined speakers, thereby outputting a leftoutput signal V_(L)(m, k) and a right output signal V_(R)(m, k) capableof recognizing a virtual multichannel signal through the pair ofheadphones.

FIG. 11 illustrates a detailed structure of a virtual localization unitwhen a sound output device is a stereo speaker.

When the sound output device is the stereo speaker different from thespeaker in FIG. 10, the virtual localization unit may additionally applya crosstalk canceller in comparison to the case of FIG. 10.

Σ and Δ of the crosstalk canceller may be derived by Equation 6. InEquation 6, H_(i) denotes a transfer function from the speaker to an earof a listener located in the same direction as the speaker. The transferfunction in such a case may be defined as a same-direction transferfunction. Further, H_(c) denotes a transfer function from the speaker toan ear of a listener located in the opposite direction to the speaker.The transfer function of such a case may be defined as anopposite-direction transfer function.

$\begin{matrix}{{\Sigma = \frac{1}{2\left( {H_{i} + H_{c}} \right)}}{\Delta = \frac{1}{2\left( {H_{i} - H_{c}} \right)}}} & \left\lbrack {{Equation}\mspace{14mu} 6} \right\rbrack\end{matrix}$

FIG. 12 illustrates a flowchart of a panoramic sound generation methodaccording to embodiments.

In operation 1201, a panoramic sound generation apparatus may calculatea panning coefficient representing directivity of a sound source usingan input signal. For example, the panoramic sound generation apparatusmay calculate the panning coefficient using a frequency component basedon the input signal.

In operation 1202, the panoramic sound generation apparatus maydetermine a direction masker for extracting a sound source of a desireddirection based on the panning coefficient. Here, the panoramic soundgeneration apparatus may determine the direction masker for masking asound source having a panning coefficient corresponding to a directionof a sound output device that corresponds to a target location.

For example, the panoramic sound generation apparatus may perform aprocess of determining a control coefficient and a process of windowing.

In further detail, the panoramic sound generation apparatus maydetermine the control coefficient for windowing the panning coefficientusing configuration information of the sound output device. For example,the panoramic sound generation apparatus may determine the controlcoefficient using a first angle based on a first sound output devicecorresponding to a left boundary and a second sound output devicecorresponding to a right boundary, a second angle based on a third soundoutput device corresponding to a target location, a third angle based ona fourth sound output device adjoining a left side of the targetlocation, and a fourth angle based on a fifth sound output deviceadjoining a right side of the target location.

In addition, the panoramic sound generation apparatus may process thepanning coefficient window based on the sound output device, using thecontrol coefficient. The panoramic sound generation apparatus mayprocess a window of which a value is maximized in a directioncorresponding to the sound output device and decreased in a directiontoward an adjacent sound output device.

In operation 1203, the panoramic sound generation apparatus may separatethe input signal as an output signal to be output to the sound outputdevice, using the direction masker. For example, the panoramic soundgeneration apparatus may apply the direction masker to the input signalbased on an angle of a sound output device corresponding to the targetlocation.

In operation 1204, the panoramic sound generation apparatus may generatea virtual signal from the output signal according to a type of the soundoutput device. For example, the panoramic sound generation apparatus maygenerate the virtual signal by applying an HRTF when the sound outputdevice is a pair of headphones. When the sound output device is a stereospeaker, the panoramic sound generation apparatus may generate thevirtual signal by applying the HRTF and a crosstalk canceller.

The methods according to the above-described embodiments may be recordedin non-transitory computer-readable media including program instructionsto implement various operations embodied by a computer. The media mayalso include, alone or in combination with the program instructions,data files, data structures, and the like. The program instructionsrecorded on the media may be those specially designed and constructedfor the purposes of embodiments, or they may be of the kind well-knownand available to those having skill in the computer software arts.

Processes, functions, methods, and/or software in apparatuses describedherein may be recorded, stored, or fixed in one or more non-transitorycomputer-readable storage media (non-transitory computer readablerecording media) that includes program instructions (computer readableinstructions) to be implemented by a computer to cause one or moreprocessors to execute or perform the program instructions. The media mayalso include, alone or in combination with the program instructions,data files, data structures, and the like. The media and programinstructions may be those specially designed and constructed, or theymay be of the kind well-known and available to those having skill in thecomputer software arts. Examples of non-transitory computer-readablestorage media include magnetic media, such as hard disks, floppy disks,and magnetic tape; optical media such as CD ROM disks and DVDs;magneto-optical media, such as optical disks; and hardware devices thatare specially configured to store and perform program instructions, suchas read-only memory (ROM), random access memory (RAM), flash memory, andthe like. Examples of program instructions include machine code, such asproduced by a compiler, and files containing higher level code that maybe executed by the computer using an interpreter. The described hardwaredevices may be configured to act as one or more software modules thatare recorded, stored, or fixed in one or more computer-readable storagemedia, in order to perform the operations and methods described above,or vice versa. In addition, non-transitory computer-readable storagemedia may be distributed among computer systems connected through anetwork and computer-readable codes or program instructions may bestored and executed in a decentralized manner. In addition, thecomputer-readable storage media may also be embodied in at least oneapplication specific integrated circuit (ASIC) or Field ProgrammableGate Array (FPGA).

Although embodiments have been shown and described, it would beappreciated by those skilled in the art that changes may be made inthese embodiments without departing from the principles and spirit ofthe disclosure, the scope of which is defined in the claims and theirequivalents.

What is claimed is:
 1. A panoramic sound generation apparatuscomprising: a panning coefficient calculation unit, using at least oneprocessor, to calculate a panning coefficient that representsdirectivity of a sound source using an input signal; a maskerdetermination unit to determine a direction masker that extracts a soundsource of a desired direction based on the panning coefficient; and achannel separation unit to separate the input signal to be used as anoutput signal, and to output the output signal to a sound output device,using the direction masker.
 2. The panoramic sound generation apparatusof claim 1, wherein the panning coefficient calculation unit calculatesthe panning coefficient using a frequency component based on the inputsignal.
 3. The panoramic sound generation apparatus of claim 1, whereinthe masker determination unit comprises: a control coefficientdeterminer to determine a control coefficient for windowing the panningcoefficient using configuration information of the sound output device;and a window processor to process a panning coefficient window based onthe sound output device using the control coefficient.
 4. The panoramicsound generation apparatus of claim 3, wherein the control coefficientdeterminer determines the control coefficient using a first angle basedon a first sound output device corresponding to a left boundary and asecond sound output device corresponding to a right boundary, a secondangle based on a third sound output device corresponding to a targetlocation, a third angle based on a fourth sound output device adjoininga left side of the target location, and a fourth angle based on a fifthsound output device adjoining a right side of the target location. 5.The panoramic sound generation apparatus of claim 3, wherein the windowprocessor processes a window of which a value is maximized in adirection corresponding to the sound output device and decreased in adirection toward an adjacent sound output device.
 6. The panoramic soundgeneration apparatus of claim 1, wherein the masker determination unitdetermines the direction masker for masking a sound source having apanning coefficient corresponding to a direction of the sound outputdevice that corresponds to a target location.
 7. The panoramic soundgeneration apparatus of claim 1, wherein the channel separation unitapplies the direction masker to the input signal based on an angle ofthe sound output device that corresponds to a target location.
 8. Thepanoramic sound generation apparatus of claim 1, further comprising avirtual localization unit to generate a virtual signal from the outputsignal according to a type of the sound output device.
 9. The panoramicsound generation apparatus of claim 8, wherein the virtual localizationunit applies a head related transfer function (HRTF) to generate thevirtual signal when the sound output device is a pair of headphones, andapplies the HRTF and a crosstalk canceller to generate the virtualsignal when the sound output device is a stereo speaker.
 10. A panoramicsound generation method comprising: calculating, using at least oneprocessor, a panning coefficient that represents directivity of a soundsource using an input signal; determining a direction masker thatextracts a sound source of a desired direction based on the panningcoefficient; and separating the input signal to be used as an outputsignal, and outputting the output signal to a sound output device, usingthe direction masker.
 11. The panoramic sound generation method of claim10, wherein the calculating calculates the panning coefficient using afrequency component based on the input signal.
 12. The panoramic soundgeneration method of claim 10, wherein the determining comprises:determining a control coefficient for windowing the panning coefficientusing configuration information of the sound output device; andprocessing a panning coefficient window based on the sound output deviceusing the control coefficient.
 13. The panoramic sound generation methodof claim 12, wherein the determining of the control coefficientdetermines the control coefficient using a first angle based on a firstsound output device corresponding to a left boundary and a second soundoutput device corresponding to a right boundary, a second angle based ona third sound output device corresponding to a target location, a thirdangle based on a fourth sound output device adjoining a left side of thetarget location, and a fourth angle based on a fifth sound output deviceadjoining a right side of the target location.
 14. The panoramic soundgeneration method of claim 12, wherein the processing processes thepanning coefficient window of which a value is maximized in a directioncorresponding to the sound output device and decreased in a directiontoward an adjacent sound output device.
 15. The panoramic soundgeneration method of claim 10, wherein the determining of the directionmasker determines the direction masker for masking a sound source havinga panning coefficient corresponding to a direction of the sound outputdevice that corresponds to a target location.
 16. The panoramic soundgeneration method of claim 10, wherein the separating applies thedirection masker to the input signal based on an angle of the soundoutput device that corresponds to a target location.
 17. The panoramicsound generation method of claim 10, further comprising: generating avirtual signal from the output signal according to a type of the soundoutput device.
 18. The panoramic sound generation method of claim 17,wherein the generating comprises: generating the virtual signal byapplying a head related transfer function (HRTF) when the sound outputdevice is a headphone; or generating the virtual signal by applying theHRTF and a crosstalk canceller when the sound output device is a stereospeaker.
 19. A non-transitory computer readable recording medium storingcomputer readable instructions to cause at least one processor toimplement the method of claim
 10. 20. A panoramic sound generationapparatus comprising: a masker determination unit to determine adirection masker that extracts a sound source of a desired directionbased on a panning coefficient which represents directivity of a soundsource based on an input signal; and a channel separation unit toseparate the input signal to be used as an output signal, and to outputthe output signal to a sound output device, using the direction masker.21. A panoramic sound generation method comprising: determining adirection masker that extracts a sound source of a desired directionbased on the panning coefficient which represents directivity of a soundsource based on an input signal; and separating the input signal to beused as an output signal using at least one processor, and outputtingthe output signal to a sound output device, using the direction masker.22. A non-transitory computer readable recording medium storing computerreadable instructions to cause at least one processor to implement themethod of claim 21.